Image processing method, image processing apparatus and computer readable storage medium

ABSTRACT

An image processing method, an image processing apparatus, an electronic device and a computer readable storage medium are provided. The image processing method includes the following. A background image and a portrait region image of a current user that a preset parameter of the background image matching the preset parameter of the portrait region image are acquired. The portrait region image and the background image are merged to obtain a merged image.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of International PatentApplication No. PCT/CN2018/105121, filed on Sep. 11, 2018, which claimspriority to Chinese Patent Application Nos. 201710813591.8,201710811471.4, 201710812528.2, 201710812662.2, 201710811814.7,201710813584.8, 201710811778.4, 201710812062.6, 201710813504.9,201710812003.9, 201710813674.7, 201710813585.2, and 201710812524.4, allfiled on Sep. 11, 2017, the entire disclosure of all of which are herebyincorporated by reference in their entireties.

TECHNICAL FIELD

The present disclosure relates to image processing technologies, andmore particularly, to an image processing method, an image processingapparatus, an electronic device and a computer readable storage medium.

BACKGROUND

An existing technology of merging a portrait and a virtual backgroundusually extracts a portrait to be merged from an image including theportrait, and adds the extracted portrait to the virtual background.

SUMMARY

The image processing method includes: acquiring a background image and aportrait region image of a current user, a preset parameter of thebackground image matching the preset parameter of the portrait regionimage; and merging the portrait region image and the background image toobtain a merged image.

The image processing apparatus according to implementations of thepresent disclosure is integrated in the electronic device. The imageprocessing apparatus includes a visible light camera, a depth imagecollection component and a processor. The visible light camera and thedepth image collection component are configured to acquire a backgroundimage and a portrait region image of a current user, a preset parameterof the background image matching the preset parameter of the portraitregion image. The processor is configured to merge the portrait regionimage and the background image to obtain a merged image.

The computer readable storage medium according to implementations of thepresent disclosure includes a computer program that may be used incombination with an electronic device capable of photographing, whereinthe computer program may be executed by a processor to implement theimage processing method as described above.

Additional aspects and advantages of embodiments of present disclosurewill be given in part in the following descriptions, become apparent inpart from the following descriptions, or be learned from the practice ofthe embodiments of the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

The above and additional aspects and advantages of embodiments of thepresent disclosure will become apparent and more readily appreciatedfrom the following descriptions made with reference to the accompanyingdrawings, in which:

FIG. 1 is a schematic flowchart illustrating an image processing methodaccording to embodiments of the present disclosure.

FIG. 2 is a schematic block diagram illustrating an image processingapparatus according to embodiments of the present disclosure.

FIG. 3 is a schematic block diagram illustrating an electronic deviceaccording to embodiments of the present disclosure.

FIG. 4 is a schematic flowchart illustrating an image processing methodaccording to embodiments of the present disclosure.

FIG. 5 is a schematic flowchart illustrating an image processing methodaccording to embodiments of the present disclosure.

FIG. 6 is a schematic flowchart illustrating an image processing methodaccording to embodiments of the present disclosure.

FIG. 7 is a schematic flowchart illustrating an image processing methodaccording to embodiments of the present disclosure.

FIG. 8 is a schematic flowchart illustrating an image processing methodaccording to embodiments of the present disclosure.

FIG. 9 is a schematic flowchart illustrating an image processing methodaccording to embodiments of the present disclosure.

FIG. 10 is a schematic flowchart illustrating an image processing methodaccording to embodiments of the present disclosure.

FIGS. 11A to 11E are schematic diagrams illustrating a measurementscenario of structured light according to embodiments of the presentdisclosure.

FIGS. 12A to 12B are schematic diagrams illustrating a measurementscenario of structured light according to embodiments of the presentdisclosure.

FIG. 13 is a schematic flowchart illustrating another image processingmethod according to embodiments of the present disclosure.

FIG. 14 is a schematic flowchart illustrating another image processingmethod according to embodiments of the present disclosure.

FIG. 15 is a schematic flowchart illustrating another image processingmethod according to embodiments of the present disclosure.

FIG. 16 is a schematic flowchart illustrating another image processingmethod according to embodiments of the present disclosure.

FIG. 17 is a schematic flowchart illustrating another image processingmethod according to embodiments of the present disclosure.

FIG. 18 is a schematic flowchart illustrating another image processingmethod according to embodiments of the present disclosure.

FIG. 19 is a schematic flowchart illustrating another image processingmethod according to embodiments of the present disclosure.

FIG. 20 is a schematic block diagram illustrating an image processingapparatus according to embodiments of the present disclosure.

FIG. 21 is a schematic block diagram illustrating an electronic deviceaccording to embodiments of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure are described below in detail andexamples of the embodiments are shown in accompanying drawings. Same orsimilar reference signs throughout the description represent the same orsimilar components or components that have the same or similarfunctions. Embodiments described below with reference to theaccompanying drawings are exemplary, intended to explain the presentdisclosure, and not construed to limit the present disclosure.

In a practical application, two images to be merged may have differencesfrom each other. As a result, a merged image obtained by simplytranslating and resizing may have a poor merging effect.

Therefore, the present disclosure provides an image processing method,an image processing apparatus, an electronic device, and a storagemedium.

An image processing method and an image processing apparatus accordingto embodiments of the present disclosure will be described below withreference to the accompanying drawings.

FIG. 1 is a schematic flowchart illustrating an image processing methodaccording to some implementations of the present disclosure. Asillustrated in FIG. 1, the method may include the following.

At block 101, a background image and a portrait region image of acurrent user are acquired. A preset parameter of the background imagematches the preset parameter of the portrait region image.

At block 102, the portrait region image and the background image aremerged to obtain a merged image.

As illustrated in FIGS. 2 and 3, the image processing method accordingto implementations of the present disclosure may be implemented by animage processing apparatus 100 according to implementations of thepresent disclosure. The image processing apparatus 100 according toimplementations of the present disclosure is integrated in an electronicdevice 1000. As illustrated in FIG. 2, the image processing apparatus100 may include a visible light camera 11, a depth image collectioncomponent 12 and a processor 20. The block 101 may be implemented by thevisible light camera 11 and the depth image collection component 12, andthe block 102 may be implemented by the processor 20.

In other words, the visible light camera 11 and the depth imagecollection component 12 may be configured to acquire the backgroundimage and the portrait region image of the current user having thepreset parameter matching the preset parameter of the background image.The processor 20 may be configured to merge the portrait region imageand the background image to obtain the merged image.

The image processing apparatus 100 according to implementations of thepresent disclosure may be integrated in the electronic device 1000according to implementations of the present disclosure. That is, theelectronic device 1000 according to implementations of the presentdisclosure includes the image processing apparatus 100 according toimplementations of the present disclosure.

In some implementations, the electronic device 1000 may include a phone,a tablet computer, a notebook computer, a smart bracelet, a smart watch,a smart helmet, smart glasses, and the like.

With the image processing method according to embodiments of the presentdisclosure, the background image and the portrait region image of thecurrent user that the preset parameter of the background image matchesthe preset parameter of the portrait region image are acquired and theportrait region image and the background image are merged to obtain themerged image. Therefore, since the preset parameter of the backgroundimage matches the preset parameter of the portrait region image, theportrait region image and the background image may be naturally merged,thereby improving visual effect of image processing.

As illustrated in FIG. 4, in some implementations, the preset parametermay include a color temperature, and the block 101 may include thefollowing.

At block 214, the color temperature of the scene where the current useris located is detected.

At block 215, an object to be merged is adjusted based on the colortemperature of the scene such that the color temperature of the objectto be merged matches the color temperature of the scene. The object tobe merged is a selected background image and/or the portrait regionimage of the current user.

At block 216, a first video image of the current user is acquired.

At block 217, multiple depth images of the current user are acquired.

At block 218, each scene image of a first scene video is processed basedon the multiple depth images, to acquire the portrait region of thecurrent user in each scene image to obtain the portrait region image.

The block 102 may include a block 219.

At block 219, each portrait region image and a corresponding backgroundimage are merged to obtain the merged image.

The blocks 214 and 215 may be implemented by the visible light camera11, the block 216 may be implemented by the depth image collectioncomponent 12, and the blocks 217, 218 and 219 may be implemented by theprocessor 20.

In an embodiment of the present disclosure, the visible light camera 11may be configured to detect the color temperature of the scene where thecurrent user is located through an internal photosensitive chip duringshooting the scene. After the color temperature of the scene isobtained, in order to well merge the portrait and the background imageto be merged, the processor 20 may be configured to adjust the colortemperature of the scene and/or the background image to be merged basedon the color temperature of the scene, such that the color temperatureof the scene matches the color temperature of the background image anddifferences of the color temperatures in the merged image are almostinvisible to human eyes. Consequently, the merging effect is good andthe user experience is improved.

In some application scenarios, for example, it is desired to hide acurrent background while the current user is in a video chat withanother, with the image processing method according to implementationsof the present disclosure, the portrait region image corresponding tothe current user and the preset background may be merged, and the mergedimage is displayed to the another.

The blocks 214 and 215 may be implemented by the visible light camera11, the block 216 may be implemented by the depth image collectioncomponent 12, and the blocks 217, 218 and 219 may be implemented by theprocessor 20.

In embodiments of the present disclosure, the visible light camera 11may be configured to detect the color temperature of the scene where thecurrent user is located through the internal photosensitive chip duringshooting the scene. After the color temperature of the scene isobtained, in order to well merge the portrait and the background imageto be merged, the processor 20 may be configured to adjust the colortemperature of the scene and/or the background image to be merged basedon the color temperature of the scene, such that the color temperatureof the scene matches the color temperature of the background image anddifferences of the color temperatures in the merged image are almostinvisible to human eyes. Consequently, the merging effect is good andthe user experience is improved.

After the color temperature of the scene is obtained, the processor 20may be further configured to acquire the color temperature of thebackground image to be merged, and compare the color temperature of thescene with the color temperature of the background image to be merged todetermine whether the color temperature of the scene matches the colortemperature of the background image to be merged. In detail, after thecolor temperature of the scene and the color temperature of thebackground image are obtained, the processor 20 is configured to comparethe color temperature of the scene and the color temperature of thebackground image to be merged to obtain a difference between the twocolor temperatures. In a case that the difference exceeds a preset colortemperature threshold, it indicates that the color temperature of thescene does not match the color temperature of the background image, andneeds to adjust the object to be merged. In some implementations, in acase that the two color temperatures do not match to each other, one orboth of the color temperatures may be adjusted. For a specificadjustment process, reference may be made to subsequent descriptions,which is not described here.

After the color temperature of the scene and/or the background image tobe merged is adjusted based on the color temperature of the scene suchthat the color temperature of the scene matches the color temperature ofthe background image, the visible light camera 11 may be configured tocontinuously shoot the scene to obtain first video images of the currentuser. In embodiments, the first video images include multiple sceneimages.

Further, the depth image collection component 12 may be configured toacquire multiple depth images of the current user. The scene image maybe a grayscale image or a color image. The depth image characterizesdepth information including depth information of each person or objectincluded in the scene of the current user. A scene range of the sceneimage is consistent with a scene range of the depth image, and eachpixel in the scene image may have respective depth informationcorresponding to that pixel provided by the depth image.

During a video shooting process, the user is usually in a moving stateand a position of the user may be changed continuously. In this case,each scene image in the first video image may correspond to a respectivedepth image. In another example, the user does not move in multiplescene images, multiple scene images in the first video image maycorrespond to a single depth image.

Further, the processor 20 may be configured to process each scene imageof the first scene video based on the multiple depth images to obtainthe portrait region of the current user in the scene image, to obtainthe portrait region image, and to merge each portrait region image and acorresponding background image to obtain a merged image.

An existing method of separating a portrait and a background is mainlyto separate the portrait and the background based on the similarity anddiscontinuity of neighboring pixels in pixel values. This method issusceptible to environmental factors such as ambient luminance. Theimage processing apparatus 100 and the electronic device 1000 accordingto embodiments of the present disclosure may obtain the depth image ofthe current user to extract the portrait region from the scene image.Since acquisition of the depth image is insusceptible to factors such asluminance or color distribution in the scene, the portrait regionextracted from the depth image may be more accurate. Particularly,boundaries of the portrait region may be accurately marked. Further, themerged image obtained by merging the accurate portrait region and thebackground may present a better effect.

With the image processing method according to embodiments of the presentdisclosure, the color temperature of the scene where the user is locatedis detected and the color temperature of the object to be merged isadjusted based on the color temperature of the scene, such that thecolor temperature of the scene matches the color temperature of thebackground image to be merged, thereby obtaining the merged image withsatisfying merging effect by merging the portrait region image extractedfrom the scene image and the background image. The color temperature istaken into consideration during the merging process, such that theportrait and the background image may be merged naturally, improve themerging effect, and improve the user experience.

In a case that the color temperature of the scene does not match thecolor temperature of the background image to be merged, one or both ofthe two color temperatures may be adjusted. In some implementations, thecolor temperature of the background image to be merged is adjusted basedon the color temperature of the scene to adjust or change the colortemperature of the background image, such that the color temperature ofthe scene matches the color temperature of the background image.

As illustrated in FIG. 5, in some implementations, the preset parametermay include the color temperature, and the block 101 may include thefollowing.

At block 214, color temperature of the scene where the current user islocated is detected.

At block 220, a virtual light source matching the color temperature ofthe scene is turned on to adjust the color temperature of the backgroundimage to be merged, such that the color temperature of the scene matchesthe color temperature of the background image.

At block 216, the first video image of the current user is acquired.

At block 217, multiple depth images of the current user are acquired.

At block 218, each scene image of the first scene video is processedbased on the multiple depth images, to acquire the portrait region ofthe current user in each scene image to obtain the portrait regionimage.

The block 102 may include a block 219.

At block 219, each portrait region image and the correspondingbackground image are merged to obtain the merged image.

In other words, in a case where the background image is adjusted basedon the color temperature of the scene such that the color temperature ofthe scene may match the color temperature of the background image, theblock 215 may include the block 220.

As illustrated in FIG. 6, in some implementations, the preset parametermay include the color temperature, and the block 101 may include thefollowing.

At block 214, color temperature of the scene where the current user islocated is detected.

At block 220, a virtual light source matching the color temperature ofthe scene is turned on to adjust the color temperature of the backgroundimage to be merged, such that the color temperature of the scene matchesthe color temperature of the background image.

At block 221, a prompt message is played upon turning on the virtuallight source.

At block 216, a first video image of the current user is acquired.

At block 217, multiple depth images of the current user are acquired.

At block 218, each scene image of the first scene video is processedbased on the multiple depth images, to acquire the portrait region ofthe current user in each scene image to obtain the portrait regionimage.

The block 102 may include a block 219.

At block 219, each portrait region image and the correspondingbackground image are merged to obtain the merged image.

In other words, in a case where the background image is adjusted basedon the color temperature of the scene such that the color temperature ofthe scene matches the color temperature of the background image, theblock 215 may include the blocks 220 and 221.

In embodiments of the present disclosure, in order to add sound inmerging images, an event of turning on the virtual light source may betaken as a listening event. When the processor 200 listens that thevirtual light source is turned on, the processor 200 is configured toadd a prompt message of turning on a switch upon turning on the virtuallight source. The prompt message may be sound or vibration.

As illustrated in FIG. 7, in some implementations, the preset parametermay include the color temperature, and the block 101 may include thefollowing.

At block 214, color temperature of the scene where the current user islocated is detected.

At block 222, color temperature of the background image to be merged isadjusted using the color temperature of the scene, such that the colortemperature of the scene matches the color temperature of the backgroundimage.

At block 216, a first video image of the current user is acquired.

At block 217, multiple depth images of the current user are acquired.

At block 218, each scene image of the first scene video is processedbased on the multiple depth images, to acquire the portrait region ofthe current user in each scene image to obtain the portrait regionimage.

The block 102 may include a block 219.

At block 219, each portrait region image and the correspondingbackground image are merged to obtain the merged image.

In other words, in a case where the background image is adjusted basedon the color temperature of the scene such that the color temperature ofthe scene matches the color temperature of the background image, theblock 215 may include the block 222.

As illustrated in FIG. 8, in some implementations, the preset parametermay include the color temperature, and the block 101 may include thefollowing.

At block 214, color temperature of the scene where the current user islocated is detected.

At block 223, a fill light is provided to the scene of the current userbased on the color temperature of the scene, such that the colortemperature of the scene matches the color temperature of the backgroundimage to be merged.

At block 216, a first video image of the current user is acquired.

At block 217, multiple depth images of the current user are acquired.

At block 218, each scene image of the first scene video is processedbased on the multiple depth images, to acquire the portrait region ofthe current user in each scene image to obtain the portrait regionimage.

The block 102 may include a block 219.

At block 219, each portrait region image and the correspondingbackground image are merged to obtain the merged image.

In other words, in a case where the color temperature of the scene isadjusted such that the color temperature of the scene matches the colortemperature of the background image, the block 215 may include the block223.

As illustrated in FIG. 9, in some implementations, the block 217 ofacquiring the multiple depth images of the current user may include thefollowing.

At block 317, structured light is projected to the current user.

At block 324, a structured light image modulated by the current user iscaptured.

At block 325, phase information corresponding to each pixel of thestructured light image is demodulated to obtain one of the multipledepth images.

As illustrated in FIG. 2, in some implementations, the depth imagecollection component 12 may include the structured light projector 121and the structured light camera 122. The block 201 may be implemented bythe structured light projector 121, and the blocks 202 and 203 may beimplemented by the structured light camera 122.

In other words, the structured light projector 121 may be configured toproject the structured light to the current user. The structured lightcamera 122 may be configured to capture the structured light imagemodulated by the current user, and to demodulate the phase informationcorresponding to each pixel of the structured light image to obtain oneof the multiple depth images.

In detail, after the structured light projector 121 projects thestructured light of a certain pattern on the face and body of thecurrent user, the structured light image modulated by the current useris formed on the surface of the face and body of the current user. Thestructured light camera 122 may be configured to capture the structuredlight image modulated and demodulate the structured light image toobtain the depth image. The pattern of the structured light may be laserstripes, Gray code, sinusoidal stripes, non-uniform speckles, and thelike.

As illustrated in FIG. 10, in some implementations, the block 325 ofdemodulating the phase information corresponding to each pixel of thestructured light image to obtain one of the multiple depth images mayinclude the following.

At block 401, the phase information corresponding to each pixel of thestructured light image is demodulated.

At block 402, the phase information is converted into depth information.

At block 403, the depth image is generated based on the depthinformation.

As illustrated in FIG. 2, in some implementations, the blocks 401, 402and 403 may be implemented by the structured light camera 122.

In other words, the structured light camera 122 may be furtherconfigured to demodulate the phase information corresponding to eachpixel of the structured light image, to convert the phase informationinto the depth information, and to generate the depth image based on thedepth information.

In detail, compared to the structured light that is not modulated, thephase information of the modulated structured light changes. Therefore,the structured light presented in the structured light image isdistorted. The changed phase information may characterize the depthinformation of objects. Consequently, the structured light camera 122may obtain the phase information corresponding to each pixel in thestructured light image through demodulation to calculate the depthinformation based on the phase information, so as to obtain the depthimage.

In order to help those skilled in the art to clearly understand theprocess of capturing the depth image of the face and body of the currentuser based on structures, the following describes a specific principleof the process with a widely-used grating projection technology (stripeprojection technology) as an example. The grating projection technologybelongs to the structured surface light in a broad sense.

As illustrated in FIG. 11A, when using structured area light forprojection, sinusoidal stripes may be generated by computer programs andprojected onto an object to be measured through the structured lightprojector 121. A curvature degree of the stripes that is modulated bythe object may be obtained by the structured light camera 122. Thecurved stripes may be demodulated to obtain the phase. The phase may beconverted into the depth information to acquire the depth image. Inorder to avoid errors or error coupling, the depth image collectioncomponent 12 needs to be calibrated before obtaining the depthinformation based on the structured light. The calibration may includecalibration of geometric parameters (for example, relative positionparameters between the structured light camera 122 and the structuredlight projector 121), calibration of internal parameters of thestructured light camera 122, calibration of internal parameters of thestructured light projector 121, and the like.

In detail, firstly, the sinusoidal stripes are produced by the computerprograms. Since in subsequent operations, it is necessary to use thedistorted stripes to obtain the phase, through for example a four-stepphase shift method, four images of stripes having the phase differenceof

$\frac{\pi}{2}$may be generated. The structured light projector 121 may project thefour images of stripes onto the object to be measured (onto the mask asillustrated in FIG. 11A) in a time-sharing manner. The structured lightcamera 122 may capture the image illustrated on the left of FIG. 11B,and simultaneously the image of stripes on a reference surface asillustrated on the right of FIG. 11B are read.

Secondly, a phase recovery is performed. The structured light camera 122may be configured to calculate the modulated phase based on the fourcaptured images of modulated stripes (i.e., structured light images).The phase image obtained here is a truncated phase image. This isbecause that the result of the four-step phase shift algorithm iscalculated by an arctangent function such that the phase of themodulated structured light is limited in a range of [−π, π]. That is,whenever the modulated phase is outside the range of [−π, π], the phasemay be converted to this range by resetting the phase. Principle valueof the resultant phase is illustrated in FIG. 11C.

In the process of the phase recovery, de-jump processing is required, toconvert the truncated phase into a continuous phase. As illustrated inFIG. 11D, the left image indicates a modulated continuous phase, and theright image indicates a reference continuous phase.

Thirdly, the modulated continuous phase is subtracted from the referencecontinuous phase to obtain the phase difference (i.e., the phaseinformation). The phase difference represents the depth information ofthe object to be measured relative to a reference surface. The phasedifference is substituted into a conversion formula (parameters involvedin the formula are calibrated) between the phase and the depth to obtaina three-dimensional model of the object to be measured as illustrated inFIG. 11E.

It should be understood that, in an actual application, depending on theapplication scenario, the structured light used in embodiments of thepresent disclosure may have any other patterns besides the above gratingpattern.

As a possible implementation, the present disclosure may also usestructured speckle light to obtain the depth information of the currentuser.

In detail, a method for obtaining the depth information with thestructured speckle light is to use a substantially flat diffractiveelement. The substantially flat diffractive element is carved with adiffraction structure that has a specific phase distribution and a crosssection of the substantially flat diffractive element has two or moreconcave-convex steps of carved structures. Thickness of a base of thediffractive element is approximately 1 micron, and heights of the stepsare not equal to each other, which may range from 0.7 microns to 0.9microns. FIG. 12A illustrates a partial diffraction structure of acollimating and beam splitting element according to embodiments. FIG.12B illustrates a sectional view along a section A-A, and units of theabscissa and the ordinate are both micros. A speckle pattern generatedby the structured speckle light has a high randomness, and the specklepattern may change with distance. Therefore, before the depthinformation is obtained using the structured speckle light, it isrequired to calibrate the speckle patterns in space. For example, withina range of 0 to 4 meters away from the structured light camera 122,planes positioned every 1 cm from the structured light camera 122 may betaken as the reference planes such that 400 speckle images may be savedafter the calibration. The shorter the distance of calibration, thehigher the accuracy of the acquired depth information. Subsequently, thestructured light projector 121 may project the structured speckle lightonto the object to be measured (i.e., the current user). Heightdifference of the surface of the object to be measured may cause changesin the speckle pattern of the structured speckle light projected ontothe object to be measured. After the structured light camera 122captures the speckle pattern (i.e., the structured light image)projected onto the object to be measured, the structured light camera122 may be configured to perform a cross-correlation operation on thespeckle pattern and the 400 speckle images saved after the calibration,so as to obtain 400 relevance images. The position of the object to bemeasured in the space may cause a peak of the relevance image. The depthinformation of the object to be measured may be obtained bysuperimposing the above peaks and performing an interpolation operation.

Multiple beams of diffracted light may be obtained after the light isdiffracted by an ordinary diffractive element. Intensities of themultiple beams of diffracted light are highly different, and a risk ofharming the human eyes is high. Even if the diffracted light isdiffracted again, uniformity of the light beam obtained is poor.Therefore, the projection effect of the light diffracted by the ordinarydiffractive element onto the object to be measured is poor. Inembodiments, the collimating and beam-splitting element may be used.This element not only has a function of light collimating onnon-collimated light, but also has a function of light splitting. Thatis, multiple beams of collimated light may exit in different directionsafter the non-collimated light reflected by a mirror passes through thecollimating and beam-splitting element. Cross-sectional areas of themultiple beams of collimated light may be approximately equal to eachother, and energy fluxes may be approximately equal to each other, suchthat the projection effect of light speckles after the light diffractionis good. In this case, light from the laser is dispersed to the multiplelight beams, thereby further reducing the risk of harming the humaneyes. Compared to other structured light of uniform arrangement, thestructured speckle light consumes less power while achieving the samecollection effect.

As illustrated in FIG. 13, in some implementations, the block ofprocessing each scene image of the first scene video based on themultiple depth images, to acquire the portrait region of the currentuser in each scene image to obtain the portrait region image may includethe following.

At block 326, the face region in the scene image is identified for eachscene image.

At block 327, the depth information corresponding to the face region isacquired from the depth image corresponding to the scene image.

At block 328, the depth range of the portrait region is determined basedon the depth information of the face region.

At block 329, the portrait region connected with the face region andhaving a depth within the depth range is determined based on the depthrange of the portrait region, to obtain the portrait region image.

As illustrated in FIG. 2, in some implementations, the blocks 326, 327,328 and 329 may be implemented by the processor 20.

In other words, the processor 20 may be further configured to identifythe face region in the scene image, to acquire the depth informationcorresponding to the face region from the depth image, to determine thedepth range of the portrait region based on the depth information of theface region, and to determine the portrait region connected with theface region and having a depth within the depth range based on the depthrange of the portrait region, to obtain the portrait region image.

In detail, a trained deep learning model may be used to identify theface region in the scene image, and to determine the depth informationof the face region based on a correspondence between the scene image andthe depth image. Since the face region includes features such as nose,eyes, ears, lips, and the like, the depth data corresponding to featuresincluded in the face region are different in the depth image. Forexample, in a case that the face is facing the depth image collectioncomponent 12, in the depth image captured by the depth image collectioncomponent 12, the depth data corresponding to the nose may be relativelysmall, while the depth data corresponding to the ears may be relativelylarge. Therefore, the depth information of the face region may be anumerical value or a numerical range. In a case that the depthinformation of the face region is a numerical value, the numerical valuemay be obtained by averaging the depth data of the face region. Inanother example, the numerical value may be obtained by taking a medianvalue of the depth data of the face region.

Since the portrait region includes the face region, i.e., depth of theportrait region and the face region are within a same depth range, theprocessor 20 may determine the depth information of the face region, setthe depth range of the portrait region based on the depth information ofthe face region, and extract the portrait region having a depth withinthe depth range and connected with the face region based on the depthrange of the portrait region, to obtain the portrait region image.

In this way, the portrait region image may be extracted from the sceneimage based on the depth information. Since acquisition of the depthinformation is insusceptible to factors such as luminance or the colortemperature in the environment, the extracted portrait region image maybe more accurate.

As illustrated in FIG. 14, in some implementations, the image processingmethod may further include the following.

At block 232, the scene image is processed to obtain a full-field edgeimage of the scene image.

At block 233, the portrait region image is corrected based on thefull-field edge image.

As illustrated in FIG. 2, in some implementations, the blocks 232 and233 may be implemented by the processor 20.

In other words, the processor 20 may be further configured to processthe scene image to obtain the full-field edge image of the scene image,and to correct the portrait region image based on the full-field edgeimage.

The processor 20 may be configured to perform edge extraction on thescene image to obtain the full-field edge image. Edges of the full-fieldedge image may include edges of the current user and edges of backgroundobjects in the scene where the current user is located. In detail, theedge extraction may be performed on the scene image by a Canny operator.The core of the edge extraction algorithm of the Canny operator mainlyincludes the followings. A 2D Gaussian filter template may be used toperform convolution operation on the scene image to eliminate noise. Adifferential operator may be used to obtain a gradient value of the grayvalue of each pixel, a gradient direction of the gray value of eachpixel may be calculated based on the gradient value, and adjacent pixelsalong the gradient direction may be found based on the gradientdirection. Each pixel is traversed. If the gray value of a pixel is lessthan gray values of two adjacent pixels along the gradient direction, itmay be considered that the pixel is not an edge point. In this way,pixels at edges of the scene image may be determined, so as to obtainthe full-field edge image after the edge extraction.

After obtaining the full-field edge image, the processor 20 may befurther configured to correct the portrait region image based on thefull-field edge image. It may be understood that the portrait regionimage is obtained by including all pixels in the scene image that areconnected with the face region and having depth within the set depthrange. In some scenes, there may be some objects that are connected withthe face region and having depth within the depth range. Therefore, inorder to make the extracted portrait region image more accurate, thefull-field edge image may be used to correct the portrait region image.

Further, the processor 20 may be configured to correct the correctedportrait region image again. For example, the processor 20 may performexpansion processing on the corrected portrait region image to expandthe portrait region image, thereby retaining edge details of theportrait region image.

After obtaining the portrait region image, the processor 20 may mergethe portrait region image and the background image to obtain the mergedimage. In some implementations, the background image may be randomlyselected by the processor 20 or selected by the current user. The mergedimage may be displayed on the display screen of the electronic device1000, or may be printed by the printer connected to the electronicdevice 1000.

Since the current user is in the video chat with the another, thevisible light camera 11 needs to capture the scene image of the currentuser in real time, the depth image collection component 12 needs tocapture the depth image corresponding to the current user in real time,and the processor 20 needs to timely process the scene image and thedepth image captured in real time to enable the another to view a smoothvideo composed of different merged images.

In a case that the color temperature of the scene does not match thecolor temperature of the background image to be merged, one or both ofthe color temperatures may be adjusted. In some implementations, a filllight may be provided to the scene based on the color temperature of thescene to illuminate the scene for adjusting or changing of the colortemperature of the scene, such that the color temperature of the scenematches the color temperature of the background image.

As illustrated in FIG. 15, in some implementations, the block 220 ofturning on the virtual light source matching the color temperature ofthe scene to adjust the color temperature of the background image to bemerged, such that the color temperature of the scene matches the colortemperature of the background image may include the following.

At block 330, a target virtual light source to be turned on isdetermined from multiple virtual light sources based on the colortemperature of the scene.

In some implementations, the multiple virtual light sources may be setfor each background image in advance. Each virtual light source has arespective color temperature. An identifier is set for each colortemperature, such that a corresponding may be established between thecolor temperature and the virtual light source and stored in theprocessor 20 in advance. After the color temperature of the scene isobtained, the processor may query the corresponding based on theidentifier of the color temperature of the scene to determine thevirtual light source matching the color temperature, take the virtuallight source determined as the target virtual light source to be turnedon.

At block 331, the target virtual light source is turned on to illuminatethe background image, to adjust the color temperature of the backgroundimage, such that the color temperature of the scene matches the colortemperature of the background image.

After the target virtual light source is determined, the processor 20may be configured to turn on the target virtual light source foremitting light. After the light reaches the background image, the colortemperature of the background image changes due to the light, such thatthe color temperature of the background image matches the colortemperature of the scene.

In some implementations, before the virtual light source that matchesthe color temperature of the scene is turned on, the processor 20 may befurther configured to determine whether the color temperature of thescene matches the color temperature of the background image. In detail,after the color temperature of the scene and the color temperature ofthe background image are obtained, the color temperature of the scene iscompared with the color temperature of the background image to be mergedto obtain a difference between the two color temperatures. In a casethat the difference exceeds a preset color temperature threshold, it isindicated that the color temperature of the scene does not match thecolor temperature of the background image, and thus the colortemperature of the background image may be adjusted.

As illustrated in FIG. 16, in some implementations, the block 221 ofplaying the prompt message upon turning on the virtual light source mayinclude the following.

At block 332, a turn-on event of the virtual light source is listened.

In embodiments of the present disclosure, a turning-on event of thevirtual light source may be set as a turn-on event in advance, and theturn-on event may be stored in the processor 20. The processor 20 maylisten to the turn-on event. When the virtual light source is turned on,i.e., the turn-on event occurs, the processor 20 may listen to theturn-on event.

At block 333, in response to listening that the virtual light source isturned on, the prompt message matching the target virtual light sourceis acquired based on an identifier of the target virtual light source.

In embodiments of the present disclosure, different virtual lightsources may correspond to different types of prompt messages. Anassociation relationship between the virtual light source and the promptmessage may be established and stored in the processor 20. The processor20 may query the above-mentioned association relationship based on theidentifier of the target virtual light source to be turned on to acquirethe prompt message that matches the target virtual light source. Theprompt message may include sound, vibration, ring and so on.

At block 334, the prompt message is played in a format corresponding tothe target virtual light source.

Different virtual light sources may correspond to different promptingformats. For example, in a case that the prompt message is a sound,different volumes and/or prompting duration may be set. After the promptmessage is determined, the processor 20 may play the prompt message inthe format corresponding to the target virtual light source.

With the image processing method according to embodiments of the presentdisclosure, by playing the prompt message upon turning on the virtuallight source, a vocal merging scene may be constructed, thereby raisinginteresting, enabling the user to know occurrence of the merging andimproving user experience.

In some implementations, before the virtual light source that matchesthe color temperature of the scene is turned on, the processor 20 mayfurther determine whether the color temperature of the scene matches thecolor temperature of the background image. In detail, after the colortemperature of the scene and the color temperature of the backgroundimage are obtained, the color temperature of the scene is compared withthe color temperature of the background image to be merged to acquirethe difference between the two color temperatures. In a case that thedifference exceeds the preset color temperature threshold, it isindicated that the color temperature of the scene does not match thecolor temperature of the background image. Consequently, both the colortemperature of the scene and the color temperature of the backgroundimage may be adjusted, such that that the color temperature of the scenematches the color temperature of the background image.

As illustrated in FIG. 17, adjusting both the color temperature of thescene and the color temperature of the background image, such that thatthe color temperature of the scene matches the color temperature of thebackground image may include the following.

At block 404, the difference between the color temperature of the sceneand the color temperature of the background image is acquired.

At block 405, the color temperature of the scene and the colortemperature of the background image are adjusted based on thedifference, to eliminate the difference between the color temperature ofthe scene and the color temperature of the background image, such thatthe color temperature of the scene matches the color temperature of thebackground image.

In detail, the processor 200 may compare the color temperature of thescene and the color temperature of the background image, to reduce thecolor temperature of the adjustment object with a larger value based onthe difference. For example, the larger vale may be reduced by half fromthe current value to obtain the color temperature of the adjustmentobject. Further, the color temperature of the adjustment object with alower value may be raised based on the difference. For example, thelower value may be raised by half from the current value to obtain thecolor temperature of the adjustment object.

For example, in a case that the color temperature of the scene is higherthan the color temperature of the background image by 2,000K, theprocessor 20 may reduce the color temperature of the scene by 1,000K andraise the color temperature of the background image by 1,000K.

In embodiments, since the difference between the color temperature ofthe scene and the color temperature of the background image is takeninto account during the adjustment process, a target color temperaturemay be closed to the color temperature of the scene, and colorcharacteristics of the original color temperature of the backgroundimage may be maintained, such that sharp changes may not occur to thebackground image. Consequently, after the portrait region image of thecurrent user is merged with the background image, the merging effect maybe better and more natural, thereby satisfying user's demands.

As illustrated in FIG. 18, in some implementations, the block 22 ofadjusting the color temperature of the background image to be mergedbased on the color temperature of the scene, such that the colortemperature of the scene matches the color temperature of the backgroundimage may include the following.

At block 335, the color temperature of the scene and the colortemperature of the background image are weighted to obtain the targetcolor temperature.

Weights for weighting color temperatures may be set for the scene andthe background image in advance, and the weights may be stored in theprocessor 20. After acquiring the color temperature of the scene and thecolor temperature of the background image, the processor 20 may obtainthe weight corresponding to the scene and the weight corresponding tothe background image respectively. The color temperature of the scenemay be multiplied by the weight of the scene to obtain a first value.The color temperature of the background image may be multiplied by theweight of the background image to obtain a second value. The first valueand the second value may be added to obtain a color temperature, i.e.,the target color temperature.

At block 336, the color temperature of the background image is adjustedto the target color temperature.

After the target color temperature is obtained, the color temperature ofthe background image may be directly adjusted to the target colortemperature. Since the difference between the color temperature of thescene and the color temperature of the background image is taken intoaccount during the weighting process, the target color temperature maybe closed to the color temperature of the scene, and colorcharacteristics of the original color temperature of the backgroundimage are maintained, such that sharp changes may not occur to thebackground image. Consequently, after the portrait region image of thecurrent user is merged with the background image, the merging effect maybe better and more natural, thereby satisfying user's demands.

As a possible implementation, the color temperature of the scene may bedirectly used as the target color temperature, and the color temperatureof the background image to be merged may be replaced with the targetcolor temperature, such that the color temperature of the backgroundimage is completely consistent with the color temperature of the scene.After the portrait region image of the current user is merged with thebackground image, the merging effect may be more natural, therebyavoiding a technical problem that the merging effect is unnatural onvision of the user.

As illustrated in FIG. 19, in some implementations, the block 233 ofproviding the fill light to the scene of the current user based on thecolor temperature of the scene, such that the color temperature of thescene matches the color temperature of the background image to be mergedmay include the following.

At block 337, a target color of the fill light is determined based onthe color temperature of the scene.

At block 338, an array of fill lights is controlled to emit light basedon the target color, such that the array of fill lights emits the lightof the target color to illuminate the scene.

As illustrated in FIG. 2, in some implementations, the blocks 337 and338 may be implemented by the processor 20.

After the color temperature of the scene is obtained, the processor 20may further obtain the color temperature of the background image to bemerged. In order to enable the color temperature of the scene to matchthe color temperature of the background image, the processor 20 maydetermine the target color of the fill lights based on the colortemperature of the scene and the color temperature of the backgroundimage. After the target color is determined, the fill light of thetarget color may be provided for the scene.

In embodiments of the present disclosure, the array of fill lights maybe further provided in the image processing apparatus 1000, and thescene may be illuminated by the array of fill lights. In detail, theprocessor 20 may control the array of fill lights to emit the light ofthe target color based on the target color. In some implementations, thearray of fill lights is an array composed of multiple fill lights.Multiple lighting strategies may be preset for the array of fill lights.The lighting strategies may be used for lights that may emit light ofdifferent colors. The lighting strategy may include on-off states ofrespective fill lights. The processor 20 may have the lightingstrategies stored therein. After the target color is determined, theprocessor 20 may match the stored lighting strategies based on thetarget color to obtain the lighting strategy corresponding to the targetcolor. The on-off state of each fill light may be controlled based onthe lighting strategy, such that the array of fill lights may emit thelight of the target color to illuminate the scene and to change thecolor temperature of the scene, such that the color temperature of thescene matches the color temperature of the background image.

In some implementations, before the fill light is provided to illuminatethe scene based on the color temperature of the scene, the processor 20may determine whether the color temperature of the scene matches thecolor temperature of the background image. In detail, after the colortemperature of the scene and the color temperature of the backgroundimage are acquired, the color temperature of the scene may be comparedwith the color temperature of the background image to be merged toobtain the difference between the two color temperatures. In a case thatthe difference exceeds the preset color temperature threshold, it isindicated that the color temperature of the scene does not match thecolor temperature of the background image, and thus the colortemperature of the scene needs to be adjusted.

As illustrated in FIG. 3 and FIG. 20, implementations of the presentdisclosure further provide an electronic device 1000. The electronicdevice 1000 may include the image processing apparatus 100. The imageprocessing apparatus 100 may be implemented by hardware and/or software.The image processing apparatus 100 may include an imaging device 10 andthe processor 20.

The imaging device 10 may include a visible light camera 11 and a depthimage collection component 12.

In detail, the visible light camera 11 may include an image sensor 111and a lens 112. The visible light camera 11 may be configured to capturecolor information of the current user to obtain the scene image of thescene. The image sensor 111 may include a color filter array (such as aBayer filter array) and there may be one or more lenses 112. During theprocess of capturing the scene image by the visible light camera 11,each imaging pixel in the image sensor 111 may sense light intensity andwavelength information from the scene to be captured to generate a setof original image data. The image sensor 111 may send the set oforiginal image data to the processor 20, and the processor 20 may obtaina color scene image after performing operations such as denoising andinterpolation on the original image data. The processor 20 may processimage pixels in the original image data one by one in various formats.For example, each image pixel may have a bit depth of 8, 10, 12 or 14bits, and the processor 20 may process each image pixel based on thesame or different bit depths.

The depth image collection component 12 may include a structured lightprojector 121 and a structured light camera 122. The depth imagecollection component 12 may be configured to capture depth informationof the current user to obtain a depth image. The structured lightprojector 121 may be configured to project structured light onto thecurrent user. The pattern of the structured light may be laser stripes,Gray code, sinusoidal stripes, or a randomly arranged speckle pattern.The structured light camera 122 may include an image sensor 1221 and alens 1222. There may be one or more lenses 1222. The image sensor 1221may be configured to capture a structured light image generated byprojecting by the structured light projector 121 onto the current user.The structured light image may be sent by the depth collection component12 to the processor 20 for processing such as demodulation, phaserecovery and phase information calculation to obtain the depthinformation of the current user.

In some implementations, functions of the visible light camera 11 andthe structured light camera 122 may be implemented by a single camera.That is, the imaging device 10 may include a single camera and a singlestructured light projector 121. The above camera may be configured tocapture a scene image, as well as a structured light image.

In addition to using the structured light to obtain the depth image, itis also possible to obtain depth image of the current user through adepth image acquisition method such as a binocular vision method, adepth image acquisition method based on time of flight (TOF) and so on.

The processor 20 may be further configured to merge the portrait regionimage and the background image to obtain the merged image. Whenextracting the portrait region image, the processor 20 may be configuredto extract a two-dimensional portrait region image from the scene imagein combination with the depth information of the depth image, or theprocessor 20 may be configured to establish a three-dimensional image ofthe portrait region based on the depth information of the depth image,and perform color filling on the three-dimensional portrait region incombination with the color information of the scene image to obtain athree-dimensional color portrait region image. Consequently, merging andprocessing the portrait region image and the background image may referto that the two-dimensional portrait region image and the backgroundimage may be merged to obtain the merged image, or the three-dimensionalcolor portrait region image and the background image may be merged toobtain the merged image.

The image processing apparatus 100 may further include an image memory30. The image memory 30 may be embedded in the electronic device 1000,or may be a memory independent from the electronic device 1000. Theimage memory 130 may include a feature of direct memory access (DMA).Raw image data collected by the visible light camera 11 or relevant dataof the structured light image collected by the depth image collectioncomponent 12 may be transferred to the image memory 30 for storing orbuffering. The processor 20 may read the raw image data from the imagememory 30 for processing to obtain the scene image, and may also readthe relevant data of the structured light image from the image memory 30for processing to obtain the depth image. In addition, the scene imageand the depth image may also be stored in the image memory 30 such thatthe processor 20 may call them for processing at any time. For example,the processor 20 is configured to call the scene image and the depthimage to extract the portrait region, and merge the extracted portraitregion image and the background image to obtain the merged image. Thebackground image and the merged image may also be stored in the imagememory 30.

The image processing apparatus 100 may further include a display 50. Thedisplay 50 may directly acquire the merged image from the processor 20or acquire the merged image from the image memory 30. The display 50 maybe configured to display the merged image for viewing by the user, orfor processing by a graphics engine or a graphics processing unit (GPU).The image processing apparatus 100 ay further include an encoder/decoder60. The encoder/decoder 60 may be configured to encode and decode imagedata of the scene image, the depth image and the merged image. Theencoded image data may be stored in the image memory 30, and may bedecompressed by the decoder and may be displayed on the display 50 afterthe decompression. The encoder/decoder 60 may be implemented by acentral processing unit (CPU), a GPU or a coprocessor. In other words,the encoder/decoder 60 may be one or more of the CPU, the GPU and thecoprocessor.

The image processing apparatus 100 further may include a control logicdevice 40. When the imaging device 10 is capturing an image, theprocessor 20 may be configured to analyze data obtained by the imagingdevice to determine statistical image information of one or more controlparameters (for example, exposure time) of the imaging device 10. Theprocessor 20 may send the statistical image information to the controllogic device 40, and the control logic device 40 may control the imagingdevice 10 to capture an image based on determined control parameters.The control logic device 40 may include a processor and/or amicrocontroller for executing one or more routines, such as firmware.The one or more routines may determine the control parameters of theimaging device 10 based on the received statistical image information.

As illustrated in FIG. 21, the electronic device 1000 according toimplementations of the present disclosure may include one or moreprocessors 200, a memory 300 and one or more programs 310. The one ormore programs 310 are stored in the memory 300 and configured to beexecuted by the one or more processors 200. The one or more programs 310include instructions for executing the image processing method accordingto any one of the implementations.

For example, the one or more programs 310 may include instructions forperforming the image processing method described as follows.

At block 101, a background image and a portrait region image of thecurrent user that the preset parameter of the background image matchesthe preset parameter of the portrait region image are acquired.

At block 102, the portrait region image and the background image aremerged to obtain a merged image.

The computer readable storage medium according to implementations of thepresent disclosure may include a computer program used in combinationwith the electronic device 1000 capable of capturing an image. Thecomputer program may be executed by the processor 200 to execute theimage processing method according to any one of the aboveimplementations.

For example, the computer program may be executed by the processor 200to implement the image processing method described as follows.

At block 101, the background image and the portrait region image of thecurrent user that the preset parameter of the background image matchesthe preset parameter of the portrait region image are acquired.

At block 102, the portrait region image and the background image aremerged to obtain the merged image.

In the description, reference throughout this specification to “anembodiment,” “some embodiments,” “an example,” “a specific example,” or“some examples,” means that a particular feature, structure, material,or characteristic described in connection with the embodiment or exampleis included in at least one embodiment or example of the presentdisclosure. The appearances of the above phrases in various placesthroughout this specification are not necessarily referring to the sameembodiment or example of the present disclosure. Furthermore, theparticular features, structures, materials, or characteristics may becombined in any suitable manner in one or more embodiments or examples.In addition, different embodiments or examples and features of differentembodiments or examples described in the specification may be combinedby those skilled in the art without mutual contradiction.

In addition, terms such as “first” and “second” are used herein forpurposes of description, are not intended to indicate or imply relativeimportance or significance, or imply the amount of this feature. Thus,the feature defined with “first” and “second” may comprise at least onethis feature. In the description of the present disclosure, “a pluralityof” means at least two, for example, two or three, unless specifiedotherwise.

Any process or method described in a flow chart or described herein inother ways may be understood to include one or more modules, segments orportions of codes of executable instructions for achieving specificlogical functions or steps in the process, and the scope of a preferredembodiment of the present disclosure includes other implementations, forexample including another order or sequence different from theillustrated order or discussed order, or including a substantiallysimultaneous order or an inverse order of functions, which should beunderstood by those skilled in the art.

The logic and/or step described in other manners herein or shown in theflow chart, for example, a particular sequence table of executableinstructions for realizing the logical function, may be specificallyachieved in any computer readable medium to be used by the instructionexecution system, device or equipment (such as the system based oncomputers, the system comprising processors or other systems capable ofobtaining the instruction from the instruction execution system, deviceand equipment and executing the instruction), or to be used incombination with the instruction execution system, device and equipment.As to the specification, “the computer readable medium” may be anydevice adaptive for including, storing, communicating, propagating ortransferring programs to be used by or in combination with theinstruction execution system, device or equipment. More specificexamples (a non-exhaustive list) of the computer readable mediumcomprise but are not limited to: an electronic connection (an electronicdevice) with one or more wires, a portable computer enclosure (amagnetic device), a random access memory (RAM), a read only memory(ROM), an erasable programmable read-only memory (EPROM or a flashmemory), an optical fiber device and a portable compact disk read-onlymemory (CDROM). In addition, the computer readable medium may even be apaper or other appropriate medium capable of printing programs thereon,this is because, for example, the paper or other appropriate medium maybe optically scanned and then edited, decrypted or processed with otherappropriate methods when necessary to obtain the programs in an electricmanner, and then the programs may be stored in the computer memories.

It should be understood that each part of the present disclosure may berealized by the hardware, software, firmware or their combination. Inthe above implementations, the multiple steps or methods may be realizedby the software or firmware stored in the memory and executed by theappropriate instruction execution system. For example, if it is realizedby the hardware, likewise in another embodiment, the steps or methodsmay be realized by one or a combination of the following techniquesknown in the art: a discrete logic circuit having a logic gate circuitfor realizing a logic function of a data signal, an application-specificintegrated circuit having an appropriate combination logic gate circuit,a programmable gate array (PGA), a field programmable gate array (FPGA),etc.

It would be understood by those skilled in the art that all or a part ofthe steps carried by the method in the above-described embodiments maybe completed by relevant hardware instructed by a program. The programmay be stored in a computer readable storage medium. When the program isexecuted, one or a combination of the steps of the method in theabove-described embodiments may be completed.

In addition, individual functional units in the embodiments of thepresent disclosure may be integrated in one processing module or may beseparately physically present, or two or more units may be integrated inone module. The integrated module as described above may be achieved inthe form of hardware, or may be achieved in the form of a softwarefunctional module. If the integrated module is achieved in the form of asoftware functional module and sold or used as a separate product, theintegrated module may also be stored in a computer readable storagemedium.

The storage medium mentioned above may be read-only memories, magneticdisks or CD, etc. Although explanatory embodiments have been shown anddescribed, it would be appreciated by those skilled in the art that theabove embodiments cannot be construed to limit the present disclosure,and changes, alternatives, and modifications can be made in theembodiments without departing from scope of the present disclosure.

What is claimed is:
 1. A method of image processing, applicable to anelectronic device, and the method comprising: acquiring a backgroundimage and a portrait region image of a current user, a preset parameterof the background image matching the preset parameter of the portraitregion image; and merging the portrait region image and the backgroundimage to obtain a merged image; wherein the preset parameter comprises acolor temperature, acquiring the background image and the portraitregion image of the current user comprises: detecting a colortemperature of a scene where the current user is located; adjusting anobject to be merged based on the color temperature of the scene suchthat the color temperature of the object to be merged matches the colortemperature of the scene; wherein the object to be merged is thebackground image and/or the portrait region image of the current user;or turning on a virtual light source matching the color temperature ofthe scene to adjust the color temperature of the background image to bemerged, such that the color temperature of the scene matches the colortemperature of the background image; or adjusting the color temperatureof the background image to be merged using the color temperature of thescene, such that the color temperature of the scene matches the colortemperature of the background image; or providing a fill light to thescene based on the color temperature of the scene, such that the colortemperature of the scene matches the color temperature of the backgroundimage to be merged; acquiring a first scene video of the current user;acquiring a plurality of depth images of the current user; andprocessing each scene image of the first scene video based on theplurality of depth images, to acquire a portrait region of the currentuser in each scene image to obtain the portrait region image; andmerging the portrait region image and the background image to obtain themerged image comprises: merging each portrait region image and acorresponding background image to obtain the merged image.
 2. The methodof claim 1, wherein adjusting the object to be merged based on the colortemperature of the scene such that the color temperature of the objectto be merged matches the color temperature of the scene comprises: inresponse to detecting that the color temperature of the scene does notmatch the color temperature of the background image, providing a filllight to the scene based on the color temperature of the scene, toadjust the color temperature of the scene such that the colortemperature of the scene matches the color temperature of the backgroundimage; or in response to detecting that the color temperature of thescene does not match the color temperature of the background image,adjusting the background image based on the color temperature of thescene, such that the color temperature of the scene matches the colortemperature of the background image; or in response to detecting thatthe color temperature of the scene does not match the color temperatureof the background image, adjusting both the color temperature of thescene and the color temperature of the background image based on thecolor temperature of the scene, such that the color temperature of thescene matches the color temperature of the background image.
 3. Themethod of claim 2, wherein adjusting both the color temperature of thescene and the color temperature of the background image based on thecolor temperature of the scene comprises: acquiring a difference betweenthe color temperature of the scene and the color temperature of thebackground image; and adjusting the color temperature of the scene andthe color temperature of the background image based on the difference,to eliminate the difference between the color temperature of the sceneand the color temperature of the background image, such that the colortemperature of the scene matches the color temperature of the backgroundimage.
 4. The method of claim 1, wherein turning on the virtual lightsource matching the color temperature of the scene to adjust the colortemperature of the background image to be merged, such that the colortemperature of the scene matches the color temperature of the backgroundimage comprises: determining a target virtual light source to be turnedon from a plurality of virtual light sources based on the colortemperature of the scene; and turning on the target virtual light sourceto illuminate the background image, to adjust the color temperature ofthe background image, such that the color temperature of the scenematches the color temperature of the background image.
 5. The method ofclaim 4, wherein determining the target virtual light source to beturned on from the plurality of virtual light sources based on the colortemperature of the scene comprises: querying a preset correspondencebetween the color temperature of the scene and the virtual light sourcebased on an identifier of the color temperature of the scene, andacquiring the virtual light source consistent with the identifier as thetarget virtual light source.
 6. The method of claim 1, furthercomprising: listening a turn-on event of the virtual light source; inresponse to listening that the virtual light source is turned on,acquiring a prompt message matching a target virtual light source basedon an identifier of the target virtual light source; and playing theprompt message in a format corresponding to the target virtual lightsource.
 7. The method of claim 1, wherein adjusting the colortemperature of the background image to be merged using the colortemperature of the scene, such that the color temperature of the scenematches the color temperature of the background image comprises:adjusting the color temperature of the background image to the colortemperature of the scene; or weighting the color temperature of thescene and the color temperature of the background image to obtain atarget color temperature; and adjusting the color temperature of thebackground image to the target color temperature.
 8. The method of claim1, before turning on the virtual light source matching the colortemperature of the scene, or adjusting the color temperature of thebackground image to be merged using the color temperature of the scene,or before providing the fill light to the scene based on the colortemperature of the scene, further comprising: comparing the colortemperature of the scene with the color temperature of the backgroundimage, to obtain a difference between the color temperature of the sceneand the color temperature of the background image and to determine thatthe difference is greater than a preset color temperature threshold. 9.The method of claim 1, wherein providing the fill light to the scenebased on the color temperature of the scene, such that the colortemperature of the scene matches the color temperature of the backgroundimage to be merged comprises: determining a target color of the filllight based on the color temperature of the scene and the colortemperature of the background image to be merged; and controlling anarray of fill lights to emit light of the target color based on thetarget color to illuminate the scene.
 10. The method of claim 9, whereincontrolling the array of fill lights to emit the light of the targetcolor based on the target color comprises: querying and acquiring alighting strategy of the array of fill lights corresponding to thetarget color based on the target color; and determining an on/off stateof each fill light in the array of fill lights based on the lightingstrategy, such that the array of fill lights emits the light of thetarget color.
 11. The method of claim 1, wherein acquiring the pluralityof depth images of the current user comprises: projecting structuredlight to the current user; capturing a structured light image modulatedby the current user; and demodulating phase information corresponding toeach pixel of the structured light image to obtain one of the pluralityof depth images.
 12. The method of claim 1, wherein processing eachscene image of the first scene video based on the plurality of depthimages, to acquire the portrait region of the current user in each sceneimage to obtain the portrait region image comprises: identifying a faceregion in the scene image for each scene image; acquiring depthinformation corresponding to the face region from the depth image;determining a depth range of the portrait region based on the depthinformation of the face region; and determining the portrait regionconnected with the face region and having a depth within the depth rangebased on the depth range of the portrait region, to obtain the portraitregion image.
 13. An apparatus for image processing, integrated in anelectronic device, and the apparatus comprising: a visible light camera;a depth image collection component, wherein the visible light camera andthe depth image collection component are configured to acquire abackground image and a portrait region image of a current user, a presetparameter of the background image matching the preset parameter of theportrait region image; and a processor, configured to: merge theportrait region image and the background image to obtain a merged image;wherein the preset parameter comprises a color temperature, and thevisible light camera and the depth image collection component areconfigured to acquire the background image and the portrait region imageof the current user by: detecting a color temperature of a scene wherethe current user is located; adjusting an object to be merged based onthe color temperature of the scene such that the color temperature ofthe object to be merged matches the color temperature of the scene;wherein the object to be merged is the background image and/or theportrait region image of the current user; or turning on a virtual lightsource matching the color temperature of the scene to adjust the colortemperature of the background image to be merged, such that the colortemperature of the scene matches the color temperature of the backgroundimage; or adjusting the color temperature of the background image to bemerged using the color temperature of the scene, such that the colortemperature of the scene matches the color temperature of the backgroundimage; or providing a fill light to the scene based on the colortemperature of the scene, such that the color temperature of the scenematches the color temperature of the background image to be merged;acquiring a first scene video of the current user; acquiring a pluralityof depth images of the current user; and processing each scene image ofthe first scene video based on the plurality of depth images, to acquirea portrait region of the current user in each scene image to obtain theportrait region image; and merging the portrait region image and thebackground image to obtain the merged image comprises: merging eachportrait region image and a corresponding background image to obtain themerged image.
 14. The apparatus of claim 13, wherein adjusting theobject to be merged based on the color temperature of the scene suchthat the color temperature of the object to be merged matches the colortemperature of the scene comprises: in response to detecting that thecolor temperature of the scene does not match the color temperature ofthe background image, providing a fill light to the scene based on thecolor temperature of the scene, to adjust the color temperature of thescene such that the color temperature of the scene matches the colortemperature of the background image; or in response to detecting thatthe color temperature of the scene does not match the color temperatureof the background image, adjusting the background image based on thecolor temperature of the scene, such that the color temperature of thescene matches the color temperature of the background image; or inresponse to detecting that the color temperature of the scene does notmatch the color temperature of the background image, adjusting both thecolor temperature of the scene and the color temperature of thebackground image based on the color temperature of the scene, such thatthe color temperature of the scene matches the color temperature of thebackground image.
 15. The apparatus of claim 13, wherein adjusting thecolor temperature of the background image to be merged using the colortemperature of the scene, such that the color temperature of the scenematches the color temperature of the background image comprises:adjusting the color temperature of the background image to the colortemperature of the scene; or weighting the color temperature of thescene and the color temperature of the background image to obtain atarget color temperature; and adjusting the color temperature of thebackground image to the target color temperature.
 16. The apparatus ofclaim 13, wherein acquiring the plurality of depth images of the currentuser comprises: projecting structured light to the current user;capturing a structured light image modulated by the current user; anddemodulating phase information corresponding to each pixel of thestructured light image to obtain one of the plurality of depth images.17. The apparatus of claim 13, wherein processing each scene image ofthe first scene video based on the plurality of depth images, to acquirethe portrait region of the current user in each scene image to obtainthe portrait region image comprises: identifying a face region in thescene image for each scene image; acquiring depth informationcorresponding to the face region from the depth image; determining adepth range of the portrait region based on the depth information of theface region; and determining the portrait region connected with the faceregion and having a depth within the depth range based on the depthrange of the portrait region, to obtain the portrait region image.
 18. Anon-transitory computer readable storage medium, comprising a computerprogram that may be used in combination with an electronic devicecapable of photographing, wherein the computer program may be executedby a processor to implement an image processing method, the methodcomprising: acquiring a background image and a portrait region image ofa current user, a preset parameter of the background image matching thepreset parameter of the portrait region image; and merging the portraitregion image and the background image to obtain a merged image; whereinthe preset parameter comprises a color temperature, acquiring thebackground image and the portrait region image of the current usercomprises: detecting a color temperature of a scene where the currentuser is located; adjusting an object to be merged based on the colortemperature of the scene such that the color temperature of the objectto be merged matches the color temperature of the scene; wherein theobject to be merged is the background image and/or the portrait regionimage of the current user; or turning on a virtual light source matchingthe color temperature of the scene to adjust the color temperature ofthe background image to be merged, such that the color temperature ofthe scene matches the color temperature of the background image; oradjusting the color temperature of the background image to be mergedusing the color temperature of the scene, such that the colortemperature of the scene matches the color temperature of the backgroundimage; or providing a fill light to the scene based on the colortemperature of the scene, such that the color temperature of the scenematches the color temperature of the background image to be merged;acquiring a first scene video of the current user; acquiring a pluralityof depth images of the current user; and processing each scene image ofthe first scene video based on the plurality of depth images, to acquirea portrait region of the current user in each scene image to obtain theportrait region image; and merging the portrait region image and thebackground image to obtain the merged image comprises: merging eachportrait region image and a corresponding background image to obtain themerged image.